Bayesian approach for randomization of heuristic algorithms of discrete programming

Discrete optimizationproblems are often solved using "heuristics" (expert opinions deening how to solve a family of problems). The paper is about ways to speed up the search by combining several heuristics involving randomization. Using expert knowledge a prior distribution of optimization results as functions of heuristic decision rules is deened and is continuously updated while solving a particular problem. This approach (BHA or Bayesian Heuristic Approach) is diierent from the traditional Bayesian Approach (BA) where the prior distribution is deened on a set of functions to be minimized. The paper focuses on the main objective of BHA that is improving any given heuristic by \mixing" it with other decision rules. In addition to providing almost sure convergence such mixed decision rules often outperform (in terms of speed) even the best heuristics as judged by the considered examples. However, the nal results of BHA depend on the quality of the speciic heuris-tic. That means the BHA should be regarded as a tool for enhancing the best heuristics but not for replacing them. The paper is concluded by a short discussion of Dynamic Visualization Approach (DVA). The goal of DVA is to exploit heuristics directly, bypassing any formal mathematical framework. The purpose of the paper is to inform the authors inventing and applying various heuristics and about the possibilities and limitations of BHA hoping that they will improve their heuristics using this powerful tool. The traditional numerical analysis considers optimization algorithms which guarantee some accuracy for all functions to be optimized. This includes the exact algorithms (that is the worst case analysis). Limiting the maximal error requires a computational eeort that in many cases increases exponentially with the size of the problem. The alternative is average case analysis where the average error is made as small as possible. The average is taken over a set of functions to be optimized. The average case analysis is called the Bayesian Approach (BA) Dia88, Moc89]. There are several ways of applying the BA in optimization. The Direct Bayesian Approach (DBA) is deened by xing a prior distribution P on a set of functions f(x) and by minimizing the Bayesian risk function R(x) DeG70, Moc89]. The risk function describes the average deviation from the global minimum. The distribution P is regarded as a stochastic model of f(x); x 2 R m where f(x) might be a